STOP Data Analysis Report

If new data comes in run:

Correct the JSON formatting bug (comma instead of bracket) for P08 and P11 with correct_ball_game_data_bug.R (only if the study is still ongoing).
sql_table_to_csv.R. Takes a while because of game data.
clean_data.R. There is a second bug for P11 that produces bad JSON game data, if you find this, you need to look for the specific row (tail+print) and then add it to line 102 of clean_data.R
generate_study_report.R
Then knit this report

Data dumping

All 4 tables aware_device, ball_game, health, medication, notification_data, consent are dumped into CSV files using sql_table_to_csv.R

Data dumping from a SQLite database

Put the SQLite database in data/raw/sqlite/pX and then run in Linux or WSL python3 Migrating_data_from_SQLite_to_MySQL.py -se ../../data/raw/sqlite

Data cleaning

Script clean_data.R does the following:

For medication
- It converts the DB timestamp into a date
- It converts the medication timestamp into a date into intake_time
For notifications
- Translate period labels from Finish to English
- Filter out buggy notifications
  - TODO Ask Yukki how notifications were logged on iOS, they are different compared to Android
  - TODO There’s a possible bug on the notificion log in Android related to evening notifications. filter_out_buggy_notifications method is taking care of this but might no be 100% precise
For diary survey
- It converts the DB timestamp into a date
For ball game
- It converts the DB timestamp into a date
- It splits the game sensors into 4 files: accelerometer, gyro, rotation, linear accelerometer, with the game date on each sensor row

Data up to August 27th

Run script generate_study_report.R to update the report table.

Considerations:

P01, P02, P03, P05, P07 are missing some data due to the certificate issue (have to update the app and sync manually). P01, 2, 4, 6, 7 have complete data now. TODO P03 data is pending to be retrived.
P05 uninstalled the app before retriving data. Data is lost.
We got P04’s data from their SQLite database. Dumped to the server using migrating_data_from_SQLite_to_MySQL.py.
Ball game JSON data was malformed for P04, P07, P08, P11, fixed using correct_ball_game_data_bug.R. 1 game had invalid information for P04 filter(X_id != 477) and 3 for P11 filter(_id != 1345 & _id != 1316 & _id != 5113).
In ball game, the linear accelerometer and rotation records for P06 sometimes didn’t have any values. Filtered those out.
We have complete data from P06: 175 games, 178 medication logs, and 65 symptom survey (two per day). For some reason, there are no notification logs TODO Ask Yukki.

Study Report
label	device_id	device_brand	start_date	games_count	medication_count	survey_count	notifications_count
Participant 01	974933df-1681-4c2d-8006-20a59327e81a	HONOR PRA-LX1	2018-07-05	168(2981)	36(2076)	30(30)	251(895)
Participant 02	87be2da7-8413-4345-bbd4-687e83c369ad	samsung SM-G955F	2018-07-08	123(128)	160(160)	27(27)	221(222)
Participant 03	900459ad-aea2-45d3-9a2b-930c677d0795	samsung GT-I9195	2018-07-07	171(295)	37(145)	37(85)	351(639)
Participant 04	19156017-dafb-437e-a371-844caf6452c6	Doro Doro 8030/8031/8028	2018-07-11	116(116)	163(165)	52(52)	512(520)
Participant 05	b26a9b78-17c9-447a-83b5-9a511ec52ae6	samsung SM-G930F	2018-07-09	160(160)	18(18)	10(10)	42(42)
Participant 06 \| iOS	e2e8a376-9dbb-4b87-9c4d-30ff11d7f22c	iPad iPad	2018-07-13	175(177)	178(178)	65(65)	0(0)
Participant 07 \| Upd 1	34ccaa7f-3ad4-4a5c-bfc9-f07ee3191b90	samsung SM-A310F	2018-08-01	53(53)	79(79)	8(8)	179(179)
Participant 08 \| p01M	e5981cc8-d738-465f-89a4-cbe9b1a54071	motorola XT1562	2018-08-02	178(182)	167(168)	38(38)	323(323)
Participant 09 \| p02M	d4cfded4-3845-4e9b-8cfd-fecf0c8b644e	iPhone iPhone	2018-08-07	105(111)	118(118)	28(28)	194(194)
Participant 10 \| p03M	8a7ca54e-7db8-458e-8b87-b024561cd719	iPhone iPhone	2018-08-07	142(337)	149(150)	43(43)	256(263)
Participant 11 \| p04M	da4b523b-55a2-4040-ae05-b8a8baf1e9e1	motorola XT1562	2018-08-09	274(274)	137(138)	36(36)	293(293)
Participant 12 \| p05M	b1c2e555-2829-40eb-a77a-d8b4b0a99751	samsung SM-J600FN	2018-08-25	68(68)	54(54)	1(1)	0(0)
Participant 13 \| p06M	66a0ff30-728c-40e3-8f42-5ca745cb3a53	iPhone iPhone	2018-08-30	74(81)	89(89)	26(27)	88(88)

Medication intake

## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.
## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.
## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.
## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.
## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.
## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.
## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.
## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.
## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.
## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.
## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.
## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.
## Scale for 'y' is already present. Adding another scale for 'y', which
## will replace the existing scale.

Notification compliance

We have to notification events:

Shown
Open

We have five notification periods:

morning (8:00 - 11:59)
noon (12:00 - 14:59)
afternoon (15:00 - 18:59)
evening (19:00 - 21:59)
dailysymptoms (10:00 - 11:00)

Need to double check this data as iOS notifications and evening notifications on Android might be wrongly processed. TODO Dailysymptoms rate is over 1.0 for P10, check with Yuuki.

The mean time plot shows the SD as well. If there is no SD whiskers is because there was only 1 notification that was open after being shown

##    participant event_count
## 1          p01  0.01209677
## 2          p02  0.11055276
## 3          p03  0.45041322
## 4          p04  0.68646865
## 5          p05  0.02439024
## 6          p06  0.00000000
## 7          p07  0.02285714
## 8          p08  0.45360825
## 9          p09  0.38738739
## 10         p10  0.65873016
## 11         p11  0.66666667
## 12         p12  0.00000000
## 13         p13  0.20588235

Diary scores

We have 5 posible scores 0-4

## geom_path: Each group consists of only one observation. Do you need to
## adjust the group aesthetic?

Game scores

Game scores are averaged within bins of 30 or 60 minutes since their last medication

Game accelerometer data

Game accelerometer data within bins of 60 minutes since their last medication

Game gyroscope data

Game gyroscope data within bins of 60 minutes since their last medication

Game rotation data

Game rotation data within bins of 60 minutes since their last medication

Game linear accelerometer data

Game linear accelerometer data within bins of 60 minutes since their last medication

Game’s mean/sd of sd of accelerometer magnitude data

Game mean/sd of the sd of accelerometer magnitude within bins of 30/60 minutes since their last medication

Game’s mean/sd of sd of gyroscope magnitude data

Game mean/sd of the sd of gyroscope magnitude within bins of 30/60 minutes since their last medication

Game’s mean/sd of sd of rotation magnitude data

Game mean/sd of the sd of rotation magnitude within bins of 30/60 minutes since their last medication

Game’s mean/sd of sd of linear accelerometer magnitude data

Game mean/sd of the sd of linear accelerometer magnitude within bins of 30/60 minutes since their last medication. TODO why are there less linear accelerometer games than accelerometer games?